Traditional on-disk row-major tables have been the dominant storage mechanism in relational databases for decades. Over the last decade, however, with explosive growth in data volume and demand for faster analytics, has come the recognition that a different data representation is needed. There is widespread agreement that in-memory column-oriented databases are best suited to meet the realities of this new world.
U.S. patent application Ser. No. 14/337,170, filed Jul. 21, 2014, entitled “Mirroring, In Memory, Data From Disk To Improve Query Performance”, (referred to hereafter as the “Mirroring Application”) is incorporated herein in its entirety. The Mirroring Application describes a dual-format database that allows existing row-major on-disk tables to have complementary in-memory columnar representations.
Various approaches have been developed for generating execution plans for queries on in-memory columnar tables compared to queries containing only row-major on-disk tables. One approach is to make no changes to the query optimizer, with the expectation that the change in table scan performance itself will make the queries perform better. Unfortunately, an execution plan generated by an optimizer designed for an on-disk row-major format may be sub-optimal on an in-memory columnar format.
Rather than making no changes to the optimizer, alternative approaches may involve (a) using simple heuristics to allow the optimizer to generate different plans, or (b) making optimizer enhancements for specific types of workloads, such as execution of star queries. However, neither of these approaches is likely to perform optimally under a variety of workloads on databases with varied schemas and different data formats running on arbitrary hardware configurations with dynamic system constraints (such as available memory).
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.